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Abstract 


We consider an interacting system of spin variables on a loopy interaction graph, identified by a 
tree graph and a set of loopy interactions. We start from a high-temperature expansion for loopy 
interactions represented by a sum of nonnegative contributions from all the possible frustration-free 
loop configurations. We then compute the loop corrections using different approximations for the 
nonlocal loop interactions induced by the spin correlations in the tree graph. For distant loopy 
interactions, we can exploit the exponential decay of correlations in the tree interaction graph to 
compute loop corrections within an independent-loop approximation. Higher orders of the ap¬ 
proximation are obtained by considering the correlations between the nearby loopy interactions 
involving larger number of spin variables. In particular, the sum over the loop configurations can 
be computed ’’exactly” by the belief propagation algorithm in the low orders of the approximation 
as long as the loopy interactions have a tree structure. These results might be useful in devel¬ 
oping more accurate and convergent message-passing algorithms exploiting the structure of loopy 
interactions. 
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I. INTRODUCTION 


The problem of computing local marginals of an arbitrary probability measure is compu¬ 
tationally hard but essential, for example, in the study of inverse problems and in solving 
for solutions to random constraint satisfaction problems. The loopy belief propagation (BP) 
algorithm is an efficient approximate algorithm that has proven very helpful in the study 
of random optimization problems [l-5|. The BP algorithm, relying on the Bethe approx¬ 
imation, is exact for systems living on tree interaction graphs. It is also expected to be 
asymptotically exact for locally tree-like graphs as long as the variables are not strongly 
correlated. In general, the accuracy and convergence of the loopy BP algorithm are not 
guaranteed, especially in the presence of short loopy interactions. Therefore, characterizing 
the algorithm performance in the presence of loopy interactions jb-S], and its generaliza¬ 


tions 


13| . have been the subject of many studies in recent years. In fact, the loopy BP 


marginals are not globally consistent in the presence of loopy interactions. This means that 
the algorithm performance can be improved by demanding more consistency for the BP 


margma 
tions 14 


s, e.g., by ensuring that the local marginals satisfy the fluctuation-response rela- 


16| . There are also many efforts to take into acconnt more accurately the effect of 


loopy interactions by an expansion around the BP solution in a loopy graph 


17 


24|. 


In this stndy, we consider the Ising model on a loopy interaction graph, e.g., the two- 
dimensional square lattice. Given a spanning tree of the interaction graph, we can unam¬ 
biguously identify the set of loopy interactions, the number of links between the endpoints of 
a loopy interaction on the spanning tree (length of loop), and the minimum number of links 
between the endpoints of two loopy interactions (distance of two loops) (see Fig. [I]). We 
write a high-temperature expansion for the loopy interactions that is given by a sum over 
all the possible frustration-free loop conhgurations. A subset of loopy interactions is called 
frustration-free if there exists at least one spin conhguration that satishes all the interactions 
in that subset. We consider the loop expansion as a partition function for a system of glob¬ 
ally interacting loop variables. The nonlocal loop interactions can be expressed in terms of 
the correlation fnnctions of the spin variables in the tree interaction graph. We then resort 
to approximations of different complexity to study such a system of globally interacting loop 
variables. See also Ref. 23| , where a high-temperature expansion of loopy interactions was 


proposed, and the low orders of loop corrections to the Gibbs free energy were computed. 
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FIG. 1. (Color online) A loopy interaction graph consisting of a spanning tree and two loopy 
interactions {ij) and {kl) of lengths Uj = 5 and l^i = 3, respectively. The two loops have distance 
dij^ki = 2 defined as the minimum separation of their endpoints along the tree. 


The following arguments rely heavily on the clustering property of the pure Gibbs state 
dehned by the probability measure of the spin variables on the tree interaction graph: the 
fact that spin correlations decay exponentially with distance along the tree. Therefore, as 
long as the loops are well separated, we can approximate the system of loopy interactions 
by an effective system of independent loops. This independent-loop (IL) approximation 
works reasonably well in the high-temperature phase down to the critical region of the 
ferromagnetic Ising model in two spatial dimensions. Here, the approximation performance 
is determined by the length and distance distribution of the loopy interactions; in one hand, 
short loops have more contribution in the loop expansion than the longer ones, and in the 
other hand, the more distant loops are better described within the IL approximation. 

To go beyond the IL approximation, we make use of the Bethe approximation to write 
the global loop interactions in terms of local correlation functions involving the neighboring 
loopy interactions. These local correlations in the tree graph can be computed exactly, 
and then the loop corrections can be computed by the standard belief propagation (BP) 
algorithm. In this way, and in the low orders of the approximation, we obtain results that 
are comparable to those of the loopy BP, with algorithms that exhibit better convergence 
properties than the loopy BP algorithm. This is because these approximations rely on the 
structure of the loopy interactions instead of the whole interaction graph. Indeed for the 
Ising model with random couplings on the two-dimensional square lattice, already the IL 
approximation works better than the loopy BP in estimating the spin correlations on loopy 
interactions. 
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The paper is organized as follows. Section [TTl gives the basic dehnitions and equations we 
use in this paper. In Sec. IIIIl we write the main equations for the high-temperature loop 
expansions. In Sec. IIVI we present approximate algorithms based on the loop expansions, 
starting from the independent-loop approximation. Finally, Sec. |V] gives the concluding 
remarks. 

II. BASIC DEFINITIONS AND EQUATIONS 

Consider the Ising model on an arbitrary graph G with Hamiltonian: 

N 

H[(t] =-'^hiCTi - ^ Jijaiaj, ( 1 ) 

i=l (P)eG 

where cr = {ai\i = 1,. .., N} with cTj G {—1, +!}• The probability of finding the system 
in spin configuration cr at inverse temperature f3 = 1/T is given by the Gibbs probability 
measure, 

( 2 ) 

The partition function Z and the free energy F are given by 

Z = (3) 

<T 

We choose a spanning tree T of the interaction graph G. This defines the set of loopy 
interactions C = {{ij) G G \ T}. The length kj of a loopy interaction (ij) G £ is defined by 
the number of links connecting the endpoints i and j on the spanning tree. The distance 
dij^ki of two loopy interactions is defined by the minimum number of links connecting the 
endpoints (i, j) to (fc, /) on the spanning tree, i.e., dij^ki = niin{4fc. In, Ijk, Iji}. Figure [2] shows 
the length and distance distributions of loops in two- and three-dimensional cubic lattices. 
We use di, d^i and doi for the neighborhood set of i in G, T and ^ = G \ T, respectively. 

Let us rewrite the Hamiltonian as H[a\ = HQ[cr] + Ffe[cr], separating the energy contri¬ 
bution of interactions on the spanning tree and the loopy interactions, 

N 

f/o[cr] = - '^hiCJi - ^ Jijdiaj, (4) 

i=l 

He[(T] = - ^ (5) 
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FIG. 2. (Color online) (a) Length and (b) distance probability distributions P{l),P{d) of the 
loopy interactions in the two-dimensional {2D) and three-dimensional {3D) square lattices with 
respect to a random spanning tree. Here L is the linear size of the lattices with periodic boundary 
conditions. 

and define 




-pHolcr] ^ g-/3Fo 


( 6 ) 


For the model on the spanning tree T, we can exactly compute the local marginals by 
the Bethe equations for the cavity marginals fi^^j{ai). Here, ^^^j{o'i) is the probability of 
hnding spin i in state (Xj, in the absence of interaction with spin j, that is, 




ap oc e 


k€doi\j \ o^k / 


( 7 ) 


These are the belief propagation (BP) equations [2, l5|]. To shorten the notation, we defined 
Bi = (3hi and Kij = 

There is only one solution to the above equations which can be obtained by iteration 
starting from arbitrary cavity marginals. Then, the local one- and two-spin marginals are 
given by 


oc n 




fiU(Ti,aj) oc .(ai)/i%(cT^) 


( 8 ) 

(9) 


At the end, the free energy is obtained by Fq = AF° — J2{ij)£T where the local free 
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energy changes read as 


,-/3AF0 ^ RiOi 




j&doi \ (Tj 




( 10 ) 

( 11 ) 


The reader can refer to Ref. for derivation of the above equations. 

The above marginals are enough to obtain the local expectation values , c°- = 

((Tjcrj)o. We can use the Bethe formalism to compute the higher order correlations 
for Xi e {0,1}, 


n 


O', 


_ gEi Ai 


( 12 ) 


where 


, E., E*e^*"* n.eaoi (E., 

e “ =-- 


E.,; n,Gaoi E., e 


jedoi 

i 7/0 


,0 




(13) 


(14) 


The are the BP cavity marginals given before, and the new cavity marginals satisfy 

the following BP equations: 


i^E,(aO oc JJ iy°^,{ak) 

k£doi\j V '^k 


(15) 


Along the same lines, one can compute the probability of observing a subset of spin variables 
in a given conhguration, q- 

The extension of the above equations to loopy graphs gives the approximate loopy BP 
equations; this is simply done by replacing d^i with di in the above equations. 

III. A HIGH-TEMPERATURE EXPANSION OF THE LOOPY INTERACTIONS 


Let us expand the partition function Z for small couplings in to get 


,-IS{F-Fo) _ 


(cosh Kij + sinh Kija^aj 




s (p)g£ \(q)6n / Q 
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where s = {sij\{ij) G £} identifies a loop configuration with Sij G {0,1} showing the absence 
or presence of an interaction. Here Uij{sij) = 6sij,o cosh Kij + sinh Kij. The averages 


n / = i: ’ ® 




(17) 


^ Yi 

0 («)6C 

can be computed efficiently, as described in the previous section. Note that these averages 
are trivial when all the nodes in the subgraph ^(s), induced by the present loopy interactions 
in s, have an even degree. 

We rewrite the right hand side of the above expansion in another form as 


,- 0 {F-Fo) ^ Kij 


^ Yl (2e ^^^sinh77,,)"“^ ( JJ 


,ai(7 


i'-' j 


• 

(ij)&C \iij)eC / Q 

with = 1 independent of the value of CTjCTj. The leading term is obtained 

when all the Sij are zero. In this way, we obtain a high-temperature expansion for the loopy 
interactions: 


^-l3{F-Fo-AFo) 


where now 


E n (n 

s {ij)ec \(p)e£ 


0 


(19) 


-PAFo ^ Yl (20) 

(U)G'C 

Let us define M+(s) = J2{ij)£C number of ferromagnetic loopy interactions 

in loop configuration s. The step function 6{Kij) = 1 for ferromagnetic loopy interactions 
{Kij > 0), otherwise 9{Kij) = 0. This allows us to separate the positive and negative 
contributions to the expansion, 

,-,iF-F,-AK, = Y. u l•’^(%)l ( n 9-^)- 

s-.M+=even {ij)eC \{ij)eC 

-EH id(^«)i(n(^^T^)'“) ■ (21) 

s-.M+=odd (ij)£C \{ij)£C / g 

We know that the right hand side is positive, and if we are interested in the free energy 
densities / = F/N in the thermodynamic limit, then 



/ — /o + ^/o + ^feven^ 


( 22 ) 


g-/9AF+e„ 


E n i'' 5 (®w)i (n 

s-.M+=even (ij)£C \{ij)GC 



(23) 
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In other words, it is that is responsible for any nonanalytic behavior of the free energy 

density. 

Note that instead of working with the ferromagnetic loopy interactions, we could work 
with the anti-ferromagnetic ones to get 

S {ij)£C \{ij)£C 

where v^j{sij) = — 1)^*^. Here the positive contribution to the expansion comes from 

loop conhgurations with an even number of anti-ferromagnetic loopy interactions M_(s) = 
Thus, in the thermodynamic limit, we have 

/ = /o — ^/o + ^fevew (25) 

g-/3AFr_ = ^ JJ I n • ( 26 ) 

s:M-=even (ij)£C \{ij)&C /q 

The two different loop expansions provide the same free energy, therefore, 

AFo +AF+=-AFo +AF", (27) 

A/o + A/g+g^ = -A/o A/g^g^. (28) 

These are indeed relationships between the free energies of two interacting systems of loop 
variables, with effective energy functions that depend on the couplings of the original 

system dehned by And, in the presence of a phase transition, the two systems display 

the same critical behavior. 

Finally, we can use both the ferromagnetic and anti-ferromagnetic representations to 
write the loop expansion as 

^ n %(%) (n ■ ( 29 ) 

s (P)e£ \(p)e£ ^ ' Q 

Now /SAFq = J2{ij)ec Fj(sp) = and we dehned ^ij = (—l)'^(^ii). Note 

that here every loop conhguration s has a nonnegative contribution to the expansion. The 
average ^ is the probability of having xtj = = i for all 

the loopy interactions. We denote this probability by Pq ~ The complexity 

of computing this probability grows exponentially with the number of connected clusters in 
the graph ^(s) induced by the set of present loopy interactions in s; the neighboring spins 








in each cluster should satisfy Kijaiaj > 0, and this is possible only if each cluster of loopy 
interactions is free of frustration, otherwise Pq = 0. In other words, only 

frustration-free loop conhgurations enter in the above expansion, and each conhguration has 
a nonnegative contribution. 


IV. APPROXIMATING THE NONLOCAL LOOP INTERACTIONS 

So far, we have made no approximation in deriving the high-temperature loop expansions 
presented in the previous section. But, to obtain the loop corrections, still we have to 
compute the nonlocal loop interactions that appear in these expansions. In this section, we 
will focus on different approximations to simplify the loop interactions. 


A. Independent-loop approximation 


For tree interaction graphs, we are sure that the averages \n( ^ are com¬ 

puted within a pure state, where clustering holds and with a good approximation for distant 
loopy interactions 


0 (u)G-C 

This independent-loop (IL) approximation results in 




(30) 




[cosh Kij + sinh Kij (uicrj),- 


(31) 


(u)G/: 


Note that in a ferromagnetic system this approximation underestimates the correlations and 
thus provides an upper bound for the free energy. 

In a homogeneous system Kij = K, and in the absence of external helds Bi = 0, we have 
((Jicrj)o = tanh^'-’'(iF), thus 


-(3F ^ -PFo + E ni In (cosh(iF) -|- sinh(iF) tanh^(iF)) , 


(32) 


where u; is the number of loopy interactions of length 1. 
In general, we obtain the magnetization by 


m = (ai) = 


d^F 


cr, 




{kl)&C 


smhKki[{ak(Ti(Ti)o - {(Tk(^i)o 
cosh Kki + sinh Kki {ukUi)n 


(33) 
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FIG. 3. (Color online) (a) The exact free energy of the ferromagnetic Ising model {hi = 0, Jij = 
1) on the 2D square lattice is compared with the independent-loop approximation based on a 
random and regular (chain) spanning tree, (b) The IL approximation compared with a mean-field 
approximation of loopy interactions (MFL), where a loopy interaction {ij) € T is replaced with 
effective fields = ^Jijrrij and = ^Jijrrii determined self-consistently. The arrows show 
the position of the ferromagnetic phase transition. 


and the two-spin correlation Qj = {cncTj) 


apF 

dKij 


by 


ft 


ft. . 

^12 ~ 


sinhiTjj -|- coshiTjj (cTjcrj)^ 


(ij) e C 


cosh Kij + sinh Kij {(Tiaj)^ 

smhKki[{(Tk(Tiaiaj)Q - {(Tkcri)^ 


{kl)£C 


cosh Kki + sinh Kki {ukai)^ 


(ij) i C. 


(34) 

(35) 


In Fig. |3]we show the free energy obtained by the IL approximation for the ferromagnetic 
Ising model on the two-dimensional square lattice. We also compare the IL approximation 
with a mean-held approximation of the loopy interactions 2^, where a loopy interaction 
JjjCrjCrj is replaced with two effective helds = ^Jijrrij and = ^Jijrrii acting on spins 
i and j. Here the local magnetizations rrii are determined self-consistently. One can indeed 
do better than this naive approximation by replacing the {rrii, rrij) in the effective helds with 
the cavity magnetizations {mi^j,mj^i) computed in the absence of the loopy interaction. 
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B. Independent-site approximation 

In the independent-loop (IL) approximation we do not consider the overlap of the neigh¬ 
boring loopy interactions. An approximate way to do this is the following: 



leading to 

^-0(F-Fo) Uij{Sij)Ylui{SgJ, (37) 

s {ij)£C i^G 

where Uij{sij) = cosh 77*^ -|- sinh 77*^, and 

Wi(saoi) = 1 + ((f^i)o - 1) (-^-)• (38) 

This independent-site (IS) approximation, which again relies on the clustering property, is 
appropriate for systems in external fields where the local magnetizations {(Ti)^ are nonzero. 

Now, the sum over the loop configurations can be computed by the Bethe approximation, 
which is exact when the loopy interactions have a tree structure. Here we need the cavity 
messages Xi^j{sij) for loop variables Sij in the absence of Uij and Uj. These cavity messages 
are governed by the following Bethe equations: 

OC 'y ^ (^ifc("Sifc)('^ifc)) • (39) 

{sik\k€doi\j} kedoi\j 

We solve the above equations by iteration, and compute the free energy from F — 7^) ~ 
AFij, where 

g-/3AFi ^ ^ Ui{sg^i) JJ (Mifc(Sifc)Afc^i(Sifc)) , (40) 

{sik\kGdoi} kGdoi 

g /3 ij _ 'y ' 'uij[Sij)Xj^i{Sij)Xi^j[Sij). (41) 

Sij 

In Fig. IHwe compare the above free energy with the exact and loopy BP free energies for 
random Ising models on the two-dimensional square lattice. We see that in the presence of 
external helds, the IS free energy is very close to the one obtained by the loopy BP, and at 
the same time, the IS algorithm converges down to very low temperatures, where the loopy 
BP does not converge. 


11 




FIG. 4. (Color online) Comparing the loopy BP (LBP) free energy with that of the independent- 
site (IS) approximation for a random Ising model {hi = ±/i, Jjj = ±1) on the 2D square lattice. 
The positive and negative values for the fields and the couplings are chosen with equal probability. 
The loopy BP algorithm does not converge for T <Ts indicated by the arrows. 


Here the local magnetizations are given by 

d/3F 


m = 




d^AF, 


dB ' ‘'u dBi 

‘ i&g ‘ 


(42) 


which can easily be computed given the \i^j{sij) and derivatives 
dUi{sQj I I - {-l)^j^5oi^ 


dBi 


- Wi)o ■ 


(43) 


Note that the Bethe free energy ~ J2(ij)£C stationary with respect to 

changes in the messages \i^j{sij), computed at the fixed point of the BP equations. Thus 
we can safely take 


d 


dXi—^j{sij ) 


AF, 

j'eg (i'j')&c 


OXi^j{sij ) 


= 0 . 


(44) 


In Appendix 1X1 we obtain the two-spin correlations Figure 0 displays the 

error we make in estimating the local magnetizations by the IL and IS approximations. The 
figure also shows the results obtained by the loopy BP algorithm, which is still doing better 
than the above approximations. In the following, we propose a systematic scheme to obtain 
more accurate approximations. 
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FIG. 5. (Color online) Comparing the RMS error in magnetization obtained by different approx¬ 
imations: independent-loop (IL), independent-site (IS), and loopy belief propagation (LBP) for a 
random Ising model with random fields and couplings uniformly distributed in (—1, -|-1), on the 2D 
square lattice. Both the IL and IS approximations are based on a maximum-weight spanning tree, 
with weight W = \Jij\- The are computed by an exact algorithm (for the smaller sys¬ 

tem in panel(a)) or by the Monte Carlo algorithm (for the larger system in panel (b)). The errorbars 
are about the size of the symbols. The RMS error is given by Am = /N. 

C. Beyond the independent-loop approximation 
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More accurate expressions for the nonlocal loop interactions can be obtained by the Bethe 
approximation, for example, 




n = 

(b)G-c 



~ n = 

(b')e£ 


on 


= 1)’ 


(45) 


Here, qij{xij = 1) and Q’idljeSoi~ respectively, the probability of having Xij = 1 

and = 1, for a given loop configuration s. More precisely, 

= (46) 

j&doi \jedoi ^ / Q 

By ^■idljgao* indeed taking into account interactions between the neighboring 

loopy interactions {ij), {ik) with distance dij^ik = 0. 

The above approximation for the nonlocal loop interactions can be used in any of the 
high-temperature loop expansions given in Sec. IIIII In Appendix [B] we show how the even 
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or odd parts of expansions [19] and [24| can be computed by an approximate message-passing 
algorithm. In the following, instead we focus on the expression given by Eq. [29l where all 
the expansion terms are nonnegative. Then we have 


^-l3{F-Fo-AFo) 


S {ij)&c iGG 


where now 


(48) 




(sp) — 'Vij(ySij')qij{Xij 1), ^i(^doi) — 




(49) 


= 1)’ 

The sum over the loop configurations can be computed within the Bethe approximation. 
Here the cavity marginal Ai^j(sij) in the absence of (vij,Vj), is recursively obtained by the 
following BP equations 


di y\j 

0 : 


^ ^ ('Gik(^ik) Ak—^i(Sik)) ■ (^ 0 ) 

{siklkGdoi\j} kedoi\j 

The solutions to these equations are found by iteration; starting from arbitrary cavity mes¬ 
sages, we update the messages according to the above equations for a sufficiently large 
number of iterations to reach a fixed point of the equations. Then, the free energy reads as 
F - Fo - AFo ^ Eieg “ E{p)e£ where 

g-/3AFi ^ ^ ViisQ^i) JJ {Vik{Sik)Ak^i{Sik)) , (51) 

{sik\kedoi} kGdoi 

g /3 = '^^v^j(^Sij)Xj^i(Sij)Xi^j(Sij). (52) 

Sij 

In Appendix we give the expressions for and Cij obtained by taking derivatives of 
the above free energy. In particular, we can easily obtain an estimation of spin correlations 
on loopy interactions, 

Cij =-^^ =-sign{Kij) --^^ {PAFi +pAFj - pAFij), {ij) e C. (53) 

Note that in computing the above derivatives, we need only to consider the explicit depen¬ 
dence of the free energy on Kij, because the Bethe free energy is stationary with respect 
to changes in the cavity messages at the fixed point. Figure [6] displays the error in spin 
correlations on loopy interactions obtained by the above correlated-loops (CL) approxima¬ 
tion for some random Ising models on the two-dimensional square lattice. For comparison. 
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FIG. 6. (Color online) Comparing the RMS errors in loopy correlations and magnetizations, 
Ac = and Am = \J~ /N^ obtained by different 

approximations: independent-loop (IL), correlated-loops (CL), loopy belief propagation (LBP), and 
generalized belief propagation (GBP) for random Ising models with random couplings uniformly 
distributed in (—1,-|-1) on the 2D square lattice. The fields are zero in panels (a) and (b), but 
they are nonzero and uniformly distributed in (—I,-|-I) in panels (c) and (d). Both the IL and 
CL approximations are based on a maximum-weight spanning tree, with weight W = \Jij\- 

The and are computed by an exact algorithm for the smaller system in panel (a), and 

by the Monte Carlo algorithm for the larger systems in panels (b)-(d). The errorbars are about 
the size of the symbols. 


the figure also shows the error obtained by the IL approximation, loopy BP algorithm, and 
generalized BP (shown only for the small system in panel (a)). The latter works with larger 
re gion s of variables (here plaquettes) to extend the consistency range of the local marginals 

0. 


25[. We see that already the IL approximation is working better than the loopy BP in 
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the absence of the external fields. And, considering the interactions between the neighboring 
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loop variables in the CL approximation results in spin correlations that are comparable to 
those of the generalized BP. Nevertheless, one could still obtain better results with the loopy 
BP for small temperatures, especially in the presence of strong helds. The same is true for 
homogeneous systems with uniform couplings, where the absence of a dominant spanning 
tree means we have to take into account the longer-ranged interactions between the loop 
variables. 

We can still improve on the above approximation by increasing the interaction range of 
the loop variables. For instance, to get better approximations, we may add interactions 
between the more distant loop variables. 


^0 1 n ^ 

,(o)G'C 


n / TT ~ 

(P)G£ jgg -LJ 


X 


n 


{ij)<{kl)&C 


Qijixij = l)qki{xki = 1 ) 


, (54) 


where 


= 1) ^ (56) 

The interactions qij^ki{xijXki = 1) can be limited to close loopy interactions with distances 
dij^ki smaller than a given distance d*. Again, from the clustering property, we expect to 
obtain a good approximation already for small d*. We leave further studies in this direction 
for future works. 


V. CONCLUSION 

We started from a high-temperature loop expansion for the Ising model on a loopy inter¬ 
action graph regarded as a globally interacting system of loop variables. We then suggested 
different approximations for the nonlocal loop interactions, writing the problem of comput¬ 
ing loop corrections in a form that is amenable to local message-passing algorithms. The 
quality of these approximations depends on the structure of the loopy interactions; when 
the loopy interactions have a tree structure, we can use the belief propagation algorithm to 
compute ’’exactly” the loop corrections within a local approximation of the nonlocal loop in¬ 
teractions. This provides approximate message-passing algorithms that rely on the structure 
of the loopy interactions, and thus exhibiting better convergence properties than the loopy 
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BP algorithm. In this way, we could obtain more accurate estimations for spin correlations 
on loopy interactions from these approximations than from the loopy BP algorithm for the 
Ising model on the 2D square lattice with random couplings. 

It would be interesting to hnd out the typical loop conhgurations close to a phase tran¬ 
sition point. The relevant loop conhgurations with no frustration could behave differently 
as we approach ferromagnetic or spin-glass phase transition points. Close to a continuous 
phase transition the loop variables are strongly correlated, and the global nature of loop 
interactions becomes more important. Therefore, it would be important to have better ap¬ 
proximations for the nonlocal loop interactions, and work with the higher-order interactions 
between the loop variables. Finally, instead of starting from a spanning tree in the loop 
expansions, we could start from any subgraph that allows for accurate computation of the 
local expectations needed in the expansions. For instance, the starting subgraph can be 
obtained by adding an appropriate subset of loopy interactions to the spanning tree [24 1. 
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Appendix A: Computing the local magnetizations and correlations 


Given the free energy F, we obtain the local expectation values from m* = (cij) = 
and Cij = {(Ti(Tj) = -§^- 

In the independent-site (IS) approximation the free energy is given by F ~ Fq -|- 
AFij, where Fq is the free energy of the system living on the span¬ 
ning tree. The other terms give the loop corrections computed by the belief propagation 
(BP) algorithm in the IS approximation. The precise dehnitions along with the computa¬ 
tion of the local magnetizations can be found in the main text. Sec. IIVBI For the two-spin 
correlations, we have 


/ \ dpAFi , 

civ ~ ^ , (//) ^ C. 


i&g 


(Al) 

(A2) 
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To compute the above quantities we need to know the BP messages Xi^j{sij) satisfying Eq. 
|39l and the following derivatives: 


dKw 

dUiiSg^i) 


= 1 cosh Kiir, (//') G C 

1 — —® 


dK, 


w 


{{aiaiav)^ - {ai)^ {cri(Tv)o) , {U') i C. 


(A3) 

(A4) 


Here the loop variables Sij = 0,1 show the absence or presence of loopy interactions, and 
the averages (• • • )o are taken in the tree interaction graph. 

In the correlated-loops (CL) approximation, the free energy reads F k, Fq + AFq + 
AT) — AFij. The precise dehnitions are given in the main text. Sec. IIV Cl 

Here we obtain a local magnetization by 

i&G {ij)&C 


mi = 


dBi 


(AS) 


where 

{sik\kedoi} HfeGao* {^ik{Sik) Xk^i{Sik)) 

. =_ 1 _ - _. 

'^Sij ) 


(A6) 

(A7) 


To compute the loop corrections, we need the cavity messages Aj^j(sjj) governed by Eq. 
and the following derivatives: 

1 — ^ijO’iO'j 


i 9qi 


n 

jedoi 


- {^i)o { n ' 

CgA* 


1 yi. 


J _ _ I ^ \Sii\ /^\ 7/7 \Si 


Qii = ^= \ 


- Wl)0 ( 


dBi 2 ' /„ v-/o\v 2 

The two-spin correlations in the CL approximation are given by 

ciu ^ -sign{Ku,) - ^ {/3AFi + 13AF, - l3AFu,) , (//') G C 

Cw ~ ^ (5c“7 (//') i £, 

i&G {ij)&C 


(AS) 

(A9) 

(AlO) 

(All) 


where 




'^{sik\k&doi} (rijGaoi^P' ~ rifcG^oi iAj{Sij)Xk^i{Sik)) 

^{sifclfcGAd rifcGA* i^iki^ik) Xk^i{Sik)) 

'^Sij (^n ~ l)-^j^i('5p')Aj_j.j,(Sjj,) 

'^Sij i^ij) ) 


(A12) 

(A13) 


18 



















In the above equations, we defined 


IV ^ 

dK,v 

IV — 

~ dKw 


(Tl(Tu 

jedoi 


1 


0 


- {(ti<tv)q (n 

^jGdoi 


1 \Si 


O'lO'V 


1 \Sii\ / (^ \Si 


' - {(^i(^v)o { (- 


0 


(A14) 

(A16) 


Appendix B: Computing the even free-energy contribution AFg+ 


In this section we explain an approximate message-passing algorithm to compute the 
contribution of loop configurations s = {sjj = 0, l|(ij) G C} with an even number of 
ferromagnetic loopy interactions, 


e ^ Yl \v±{sij)\/ JJ . (Bl) 

s-.M+=even \{ij)€C / q 

Here M+(s) = X](p)e£Heaviside function, and 'yj^(sp) = — 1). 

The aim is to find an estimation of by the Bethe approximation, given an approximate 

factorization of the nonlocal loop interactions 



Here Xij = ^ Qij{xij = 

interaction graph. Similarly, QiiYljedoi 
Then we have 


QijiXij 


TT ^yTijedpi^ij 


(B2) 


1) is the probability of having Xij = 1 in the tree 
Xij = 1) is the probability of having ~ 


,-/3AFe+e 


E n 

s:M^=even (ij)£C 


Vij[Sij 




ieg 


with 


(%) = \vtjisij)\qij{xij = 1 ), = 


= 1) 


(B3) 


(B4) 


Uj(,a,iq^j{x^j = !)■ 

Let us for simplicity assume that the graph Q induced by the loopy interactions is a tree. 


otherwise we choose one of its spanning trees 2a, l27|. Then, we introduce auxiliary variables 
Ti^j G {—1, -1-1} on the directed links of G, where Ti^j = (—1)^+ gives the parity of the 
ferromagnetic loopy interactions in the cavity tree Gi^j] a branch of the tree that includes 
i and cavity trees {Gk^i\k G \j}. More precisely. 


Ti^j 


k£doi\j 


(B5) 
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Now we have to work with extended marginals Xi^j{sij,Ti^j) governed by the following BP 
equations 



Pc— J-i)) • (®6) 


The solution to these equations can be found by iteration. Then, for an even number of 
ferromagnetic loopy interactions, the free energy shifts are given by 






{sik,'rk^i\kedoi} 



where Tij = (—and the local Uj is computed like the cavity one Ti^j but 
considering the contribution from all the neighbors in doi. Finally, the even free energy reads 
asAf+„„ = E.AF.-EB,«AF.,. 
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